1. Field of the Invention
The present invention relates, in general, to storage systems such as those optimized for delivering streaming data (e.g., a video image storage depository) and to copying and storing data within data storage systems, and, more particularly, to a method for creating replicas within a media storage system without introducing additional load, e.g., without introducing additional READ streams from the storage system or device as is the case with conventional COPY commands or explicit COPY operation.
2. Relevant Background
The computer industry has seen a huge growth in the use of streaming data or streaming media to deliver content to users and customers connected to a network, such as the Internet or a local or wide area network. For example, audio and video media is delivered in a “stream” from a server so that the end user does not have to wait several minutes or longer to download multimedia files. Files are transmitted over IP networks and then viewed on-the-fly as received by the client computer or network device. It has become very common for live or pre-recorded audio/video content to be encoded (digitized and compressed) and then transmitted sequentially over the Internet or an intranet as packets of information. Data is received and decompressed by the end-user using a player on their network device or client computer that may be integrated into the browser or provided as a separate application. With the development of streaming technologies, people can simply connect to a network, select an audio or video object, and then listen or view the media or data as it streams to their network or computer device.
With this large demand for streaming data or media, a number of problems have developed that can cause problems with delivering the requested data in a timely manner or without interruption (e.g., at a smooth delivery rate required for a quality viewing or listening experience). For example, streaming media storages systems or depositories may be created to support delivery of requested streaming data or data objects. However, during operation of such a streaming media storage system, additional replicas of data objects often have to be created dynamically to meet fluctuations or increases in the demand for concurrent access to the data objects (e.g., peaks in demands for a particular song or movie data object). Conventional methods of simply copying the data object present in a storage device is not satisfactory because it creates additional burdens on the already overloaded source storage device as the copying makes it more difficult to access the data object and increases demand for concurrent access.
To better understand problems associated with effectively managing the storage and delivery of streaming media, it may be useful to look at that streaming data and problems associated with its delivery. Streaming data typically refers to data objects that are large in size, e.g., many megabytes to many gigabytes or larger in size and that are accessed in a continuously streaming fashion, i.e., the bytes making up the data object are accessed and transmitted to an end user in order of their logical location within the data object. Common examples of streaming data, as discussed above, are digital video files and digital audio files, which are typically static files (i.e., read-only content) such as a pre-recorded movie or music program or song. To provide a pleasing viewing and/or listening experience, the data needs to be presented to the user's device (and to the human user) in a smoothly streaming manner. From the perspective of the data storage system serving the data (e.g., the media depository), this requirement indicates the need to supply the data stream from one of its storage devices (e.g. the source data storage device) to the user's device (e.g., a video terminal or the like) at a guaranteed, predetermined minimum rate, such as at a rate that is more than a set amount of bits per second.
Maintaining high access to a wide range of data objects while maintaining the predetermined minimum access or streaming rate is difficult. In this regard, a data storage system servicing streaming media or data may include a number of storage devices each of which is capable of supplying a certain number of data streams concurrently while meeting the rate guarantees or desired streaming rate. As a result, the number of concurrent streams, e.g., read output streams, that a data storage system can support for a given data object, such as popular movie or song, is dependent on the number of streams each device in the system can support and on the number of the devices in the system that include a copy of the given data object. This number of concurrent streams may also be reduced by serving other data streams for other data objects (e.g., movies or songs). In general, the larger the number of devices that hold a copy of a data object the more concurrent streams of that object that can be served by the data storage system.
However, because each additional copy or replica of the data object consumes additional and limited capacity of the system, it is generally desirable to maintain just enough copies or replicas of the data object located on just enough storage devices to provide the number of concurrent streams to meet existing user demand. For example, a system manager may decide to have copies of a popular or high demand movie kept on several storage devices while only keeping one copy or fewer copies of a not as popular or low demand movie in order to avoid wasting storage capacity in the system. However, user demand for a data object fluctuates, and sometimes largely, over time, and the number of concurrent streams needed often will change.
To optimize performance and capacity usage, one management approach is to change the number of copies of the data object with current or predicted demand, e.g., if demand is rising more copies are made and when demand subsides copies are deleted. For example, a system may be configured such that when the system detects that a particular storage device is becoming overloaded with too many concurrent streams being served for a particular data object (e.g., movie or song or other data object) the system acts to create a replica of the data object in a less busy or idle storage device. A common and straight forward approach to creating a replica of a data object is to have a system or data storage management application issue a command to copy (i.e., a COPY command) the data object to an idle or less busy device in the system. This conventional replication technique is problematic, however, because performing the COPY command results in yet another stream from the already busy device to retrieve the data object to be written to the other storage device. To address this problem, some data storage systems have attempted to better predict upcoming demand so as to avoid overloading the storage device by initiating the COPY operations long before the storage device has reached its capacity. This technique also has its problems as many times the prediction of increased demand is wrong and a COPY operation is performed when the copy or replica is never actually used or needed (i.e., the original storage device does not reach its capacity), which results in wasting storage space and using system bandwidth that could be used for other more useful operations.
Hence, there remains a need for an improved method and system for creating replicas or copies of data objects or streaming data in a data storage system that is serving multiple, concurrent streams of data. Such a method and system preferably would not create an additional load on storage devices storing the object being replicated (i.e., the source device).